Visualization Techniques for Surveillance: Visualizing What Cannot Be Seen and Hiding What Should Not Be Seen

Introduction Video surveillance (closed-circuit television, CCTV) systems are becoming ubiquitous; you can hardly take a walk in a modern city center without being recorded by surveillance cameras, even less so inside shops and malls. A large shopping mall can have hundreds of cameras installed, a football stadium maybe one hundred. Typically, these cameras are connected to a control center, where a subset of the camera views is displayed on a number of screens or video walls, such as in figure 1. Human operators monitor the imagery, trying to assess the situation and detect incidents and threats. There are several problems associated with this work situation, partly due to the difficulty for the human operator to interpret multiple channels of visual information, partly due to the limitations of what the human visual system is able to see, and partly due to the fact that we do not want surveillance operators to see everything. The aim of this paper is to categorize, describe and discuss surveillance problems that can be addressed by exploiting visualization techniques. The problems, solutions, examples and illustrations in this paper are the results of interviews and discussions with various security and technology experts, such as security managers at sports arenas, public transport systems, and airports, as well as scientists in sensor technologies outside the author's own expertise. While most of the results from those discussions are out of scope of this paper (i.e., they were issues of security and not of visualization), a number of existing and emerging visualization techniques were brought up. This paper is an attempt to categorize, describe and discuss them, and also to make predictions about their future use in surveillance as well as in our daily lives. Four such problems are selected, three that are associated with the main problems of surveillance technology in general (detection, situation awareness, privacy), and one where visualization is the critical component (visualizing the invisible). The paper is written from an engineering perspective, aiming at explaining for the non-technical reader how advancements in computer graphics, sensor and surveillance technologies lead to problems (and solutions) that to their nature are of an ethical, legal or perceptive nature rather than technological. Focus is on visualization solutions, while ethical/privacy/legal issues are briefly mentioned and technology described only as much as necessary for understanding the treated concepts. Regarding technology, much of this paper involves automatic (computerized) analysis of images, that is, a …